Speech Recognition of South African English Accents

Declaration By submitting this thesis electronically, I declare that the entirety of the work contained therein is my own, original work, that I am the sole author thereof (save to the extent explicitly otherwise stated), that reproduction and publication thereof by Stellenbosch University will not infringe any third party rights and that I have not previously in its entirety or in part submitted it for obtaining any qualification. Summary Several accents of English are spoken in South Africa. Automatic speech recognition (ASR) systems should therefore be able to process the different accents of South African English (SAE). In South Africa, however, system development is hampered by the limited availability of speech resources. In this thesis we consider different acoustic modelling approaches and system configurations in order to determine which strategies take best advantage of a limited corpus of the five accents of SAE for the purpose of ASR. Three acoustic modelling approaches are considered: (i) accent-specific modelling, in which accents are modelled separately; (ii) accent-independent modelling, in which acoustic training data is pooled across accents; and (iii) multi-accent modelling , which allows selective data sharing between accents. For the latter approach, selective sharing is enabled by extending the decision-tree state clustering process normally used to construct tied-state hidden Markov models (HMMs) by allowing accent-based questions. In a first set of experiments, we investigate phone and word recognition performance achieved by the three modelling approaches in a configuration where the accent of each test utterance is assumed to be known. Each utterance is therefore presented only to the matching model set. We show that, in terms of best recognition performance, the decision of whether to separate or to pool training data depends on the particular accents in question. Multi-accent acoustic modelling, however, allows this decision to be made automatically in a data-driven manner. When modelling the five accents of SAE, multi-accent models yield a statistically significant improvement of 1.25% absolute in word recognition accuracy over accent-specific and accent-independent models. In a second set of experiments, we consider the practical scenario where the accent of each test utterance is assumed to be unknown. Each utterance is presented simultaneously to a bank of recognisers, one for each accent, running in parallel. In this setup, accent identification is performed implicitly during the speech recognition process. A system employing multi-accent acoustic models in this parallel configuration is shown to achieve slightly improved performance relative to …